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INTERNATIONAL ADULT LITERACY AND BASIC SKILLS SURVEYS IN THE OECD 

REGION 


Abstract 


Both within and beyond the OECD region, governments and other stakeholders are increasingly interested 
in the assessments of the skills of their adult populations in order to monitor how well prepared they are for 
the challenges of the knowledge based society. The current paper provides an overview of the two 
international assessments of adult literacy which have already taken place in the OECD region - the 
International Adult Literacy Survey (IALS) and the Adult Literacy and Life Skills Survey (ALL) as well as 
of the forthcoming OECD Programme for the International Assessment of Adult Competencies (PIAAC). 
The conceptual framework for the assessments is described with a focus on the links between the different 
assessments. In addition, the paper provides a survey of the outputs of IALS and ALL including a review 
of the major themes addressed in the literature which has used data from these surveys as well as a brief 
discussion of their policy impact. 

The author would like to thank his colleagues Stephan Vincent-Lancrin and Andreas Schleicher as well as 
Yvan Clermont from Statistics Canada for their useful comments on earlier versions of this paper. Errors 
and omissions remain the responsibility of the author. Lionnuala Canning and Niccolina Clements 
provided essential editorial assistance. 

Resume 

Au sein et au-dela de la region OCDE, les gouvernements et les autres parties prenantes sont de plus en 
plus interesses par revaluation des competences de leur population adulte afin de verifier si ils sont bien 
prepares ou non aux defis de la societe basee sur la connaissance. Le present document fournit un aper 5 u 
de deux evaluations internationales sur P alphabetisation des adultes qui ont ete effectuees dans la region 
OCDE : l'Enquete internationale sur l'alphabetisation des adultes (ELAA) et l’Enquete sur la litteratie et les 
competences des adultes (ELCA) ainsi que le Programme, prochainement disponible, pour T evaluation 
internationale des competences des adultes (PIAAC). La structure conceptuelle des evaluations est decrite 
comme etant centree sur les liens entre les differentes evaluations. De plus, le document fournit une 
enquete sur les resultats de l’EIAA et l’ELCA. Cette derniere comprend une vue d’ensemble des themes 
majeurs de l’alphabetisation et ce, en utilisant les donnees de ces enquetes ainsi qu’un bref debat sur 
l’impact de leur politique. 

L’ auteur souhaite remercier ses collegues Messieurs Stephan Vincent-Lancrin et Andreas Schleicher ainsi 
que Monsieur Yvan Clermont de Statistics Canada pour leurs commentaires utiles sur les versions initiales 
de ce document. Les erreurs et omissions sont de la responsabilite de 1’ auteur. Lionnuala Canning et 
Niccolina Clements ont beaucoup aide pour T edition. 
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1. Introduction 


Since the early 1990s, two large scale cross-country assessments of the literacy and basic skills of the 
adult population have taken place. The first was the International Adult Literacy Survey (LALS) which was 
implemented over the period 1994-1998. The second, known as the Adult Literacy and Life Skills Survey 
(ALL), was undertaken over 2002-2006. A successor to LALS and ALL - the Programme for the 
International Assessment of Adult Competencies (PIAAC) - is currently being developed under the aegis 
of the OECD 1 . 

The objective of this paper is to provide an overview of the objectives, design, use and impact of the 
international adult literacy and basic skills surveys implemented in the OECD region and, more generally, 
in the advanced economies. Section 2 provides a brief history of the development of international adult 
skills surveys. Section 3 outlines the conceptual framework underlying IALS, ALL and PIAAC. Section 4 
describes the main developments between IALS, ALL and PIAAC. In Section 5, an overview is provided 
of the use of data from IALS and ALLS. The policy impact of these surveys is briefly covered in Section 6 
and some concluding observations are presented in the seventh and final section. 

2. International adult skills surveys: a brief history 

The origins of the research programme that began with IALS and continues with the development of 
PIAAC lie to a large extent in pioneering work on the large scale assessment of young people and adults 
undertaken in the 1980s and early 1990s by the (US) Educational Testing Service (ETS) and Statistics 
Canada. Surveys conducted in the US such as the 1985 Young Adult Literacy Survey (YALS), a survey of 
job seekers, and the 1992 National Adult Literacy Survey or (NALS) demonstrated that it was possible to 
combine advances in psychometrics, reading theory and large scale assessment with household survey 
methodologies. 2 The 1989 Canadian Survey of Literacy Skills Used in Daily Activities (LSUDA) extended 
the methods used in the US assessments to a multi-lingual situation and demonstrated that they were able 
to provide comparable information regarding the literacy skills of the English and Lrench speaking 
populations in Canada (Montigny, et al., 1991). 

The results of the US and Canadian assessments generated considerable interest in terms of the policy 
relevance of their findings and the possibilities that they opened regarding the collection of robust 
empirical data regarding literacy performance at the population level. Lor example, a 1992 OECD report 
(Benton & Noyelle, 1992) argued that the findings of the US and Canadian assessments which 
demonstrated that a large share of the population had low levels of literacy and that the problem was 
spread across the population had important implications for education and training policy in the advanced 
economies. It was also noted that the evidence of the scale of the literacy problem in most countries was 
patchy and that better data was needed for policy analysis. The authors advocated that countries which had 
not already done so should conduct direct assessments of literacy (1992, p.57) and suggested that 
international organisations such as the OECD could play a role in promoting cross-national efforts in the 
analysis and measurement of literacy. 


1 UNESCO is developing an instrument known as the Literacy Assessment and Monitoring Programme (LAMP) 

designed for use in less developed economies. Information regarding LAMP can be found at 
http://www.uis.unesco.org/ev.php7URL ID=6409&URL DO=DO TOPIC&URL SECTIQN=201 

2 See Kirsch and Murray (1998) for a useful summary, and also Benton and Noyelle (1992, pp. 23-27). 


5 



EDU/WKP(2009)5 


Statistics Canada and ETS worked together with a group of nine countries and the OECD in the early 
1990s to develop IALS instruments and procedures. Data collection took place in 1994 3 . Two subsequent 
waves of data collection took place using the IALS instrumentation, the first in 1996 (with five countries 
taking part) and the second in 1998 (with a further nine countries). In all, 22 countries administered IALS 
between 1994 and 1998. In addition several countries undertook national surveys based on IALS. 

In parallel with the further waves of IALS data collection, Statistics Canada sponsored work on the 
development of a successor to IALS with the goal of measuring a broader range of adult skills than had 
been covered in IALS. An ambitious development agenda was proposed for the new survey (ALL) which 
included the elaboration of conceptual frameworks and possible measures in seven skill domains: prose 
literacy, document, literacy, numeracy, teamwork, problem-solving, practical cognition, and working with 
information technology (Murray, et al., 2005, p. 14). 4 While frameworks were developed for all of the 
proposed skill domains, it did not prove possible to construct measures of sufficient reliability for practical 
cognition 1 teamwork, and information and communication technology. Measures of attitudes to teamwork 
and familiarity and use of ICTs were incorporated into the ALL Pilot background questionnaires but only 
the questions relating to familiarity with and use of ICT were retained for the main ALL study (2005, 
P-15). 

Seven countries were involved in the initial round of ALL in 2002-2003. A further five countries 
undertook the assessment in 2006. 

In 2003, the OECD began to explore the possibility of developing an international adult skills 
assessment. Work continued over 2004 and 2005 with the release in October 2005 of a document outlining 
a strategy for the proposed PIAAC (OECD, 2005; Schleicher, 2008). This proposed an ongoing cycle of 
assessments stalling in 2009 or 2010. The suggested content for the first assessment included measurement 
of literacy in an ICT context, measurement of basic reading ‘component’ skills, and (if it proved valid and 
reliable in a cross-country context) a module concerning skills use in the workplace. In addition, the 
strategy proposed that there be options for the over-sampling of youth and/or the older (50-65 years of age) 
population. The assessment was conceived as being computer based. In subsequent cycles it was envisaged 
that the range of competencies assessed could be broadened and that there could be an option of a linked 
employee/employer survey. 

Work on PIAAC began in earnest in the second half of 2007 with the release, in late November, of a 
call for tender for a contractor or contractors to undertake the international development component of 
PIAAC. This was based around a timetable which envisaged a field test in 2010, the main data collection 
in 201 1 and reporting of results in 2013. In early 2008, a steering body for the project was established (the 
Board of Participating Countries or BPC) and the international contractor for PIAAC was appointed - a 

3 This first round of IALS was not without controversy. When results became available in 1995, France was found to 

have far higher proportions of its population performing at the lowest proficiency levels (with 75 percent of 
the population estimated to be the two lowest levels in the case of the prose scale) than the other 
participating countries with the exception of Poland. These results were contested by the French authorities 
(Bottani & Vrignaud, 2005, p.39; Kirsch & Murray, 1998, pp. 20-21). An independent review of the quality 
of the data in IALS was undertaken which found that problems in the design and implementation of the 
study threatened ‘the validity of any comparisons of literacy levels across countries’ (Kalton, et al., 1995, 
p. 14) . The seriousness of the problems identified by the independent review was contested, however, by 
the project organisers. Murray, et al., (1998) acknowledge a number of shortcomings with survey processes 
but find no evidence that these had seriously compromised data quality. Results of the first wave of IALS, 
including rankings but excluding French data, were published in 1995 (OECD and Statistics Canada, 
1995). 

4 The possibility of including a sample of workers in firms and the use of computer testing in the assessment were 

initially canvassed, these options were rapidly ruled out on pragmatic grounds (Murray, et al., 2005, p. 1 3). 
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consortium of US and European organisations led by ETS. As at March 2009, 28 countries intended to 
participate in PIAAC. 


Data collection waves and survey participants by survey 

Survey 

Date of data 
collection 

Participants 




IALS (wave 1) 

1994 

Canada, Germany, France, Ireland, the Netherlands, Poland, 
Sweden, Switzerland (French and German speaking), USA 

IALS (wave 2) 

1996 

Australia, the Flemish Community in Belgium, Great Britain, 
New Zealand, Northern Ireland 

IALS (wave 3) 

1998 

Chile, Czech Republic, Denmark, Finland, Hungary, Italy, 
Norway, Slovenia, Switzerland (Italian speaking) 

ALLS (wave 1) 

2002 

Canada, Bermuda, Italy, Mexico (Nuevo Leon) Norway, 
Switzerland, USA 

ALLS (wave 2) 

2006 

Australia, Netherlands, New Zealand, Korea, Hungary 

PIAAC 

2011 (planned) 

Australia, Austria, Belgium, Canada, Chile, Cyprus, Czech 
Republic, Denmark, Estonia, Finland, France, Germany, 
Hungary, Ireland, Italy, Japan, Korea, Malta, the Netherlands, 
Norway, Poland, Portugal, Slovakia, Slovenia, Spain, Sweden, 
United Kingdom, USA 



3. Measuring literacy: the Conceptual Framework of IALS, ALLS and PIAAC 

IALS, ALL and PIAAC share a common conceptual framework and approach to the assessment of 
literacy skills (conceived broadly) covering the conceptualisation of literacy, the approach to measurement, 
data quality and reporting of results. The basic elements of this framework are outlined below. 

Conceptualising Literacy 

Literacy in its relationship to society and the economy 

Literacy skills (and basic cognitive skills more generally) are conceived as an important determinant 
of the life chances of individuals and of social and economic well-being at the level of nations. Beyond 
providing information on the levels and distribution of skills in the population, the policy focus of these 
studies has been on the relationship of observed skills to economic and social success, the identification of 
at risk populations and analysis of the causes and consequences of the observed skill distributions (Murray, 
et al., 2005, pp.16-17; Schleicher, 2008). 

Literacy conceived as a complex mode of adult behaviour 

‘Literacy’ and other basic skills are conceived as social practices involving a complex set of 
cognitive and information processing skills required to use a variety of text types for a range of purposes in 
a range of contexts. For example, in IALS and ALL, literacy is defined as ‘using printed and written 
information to function in society, to achieve one’s goals and to develop one’s knowledge and potential’ 
(OECD & Statistics Canada, 1995, p. 14). PIAAC has adopted a similar definition, amended to place 
greater emphasis on a wider variety of text types. 
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The consequence of this for the design of assessments in LALS and its successors has been that: 

• Each skill domain is conceived as a multidimensional construct 

• Assessment tasks cover a broad range of difficulty 

• Tasks cover the range of cognitive processes believed to influence task difficulty, and 

• Assessment tasks and stimulus materials are drawn from a wide range of life contexts. 

It is important to note that the focus of IALS, ALL and PIAAC is on certain aspects of literacy, in 
particular, the understanding and use of texts. Writing skills and the ability to produce or format documents 
is not assessed. This is not because these skills are not considered as important aspects of literacy 
considered broadly, but, in large part, because of the practical difficulties associated with the assessment of 
adults’ writing in large scale international surveys. 

Measurement 

Measuring the skills of the population 

The objective of the assessments is to describe the level and distribution of the skills of the adult 
population, not to test the proficiency of individuals. The total number items used in the assessments is 
larger than the number taken by any single respondent, each of whom who undertakes a subset of the tasks 
administered. A rotated block design is used which involves an overlapping subset of items being 
administered to respondents. At the same time, each separate test item is taken by a representative sample 
of adults. 

Comparability across languages and cultures 

The assessments are explicitly developed as international or cross-national assessments designed to 
provide valid and reliable measures of proficiency across different countries, languages and cultures. A 
considerable investment is, therefore, made in ensuring that the assessments are, as far as is possible, 
culturally and linguistically unbiased - in other words, to ensure that a person of the same ability will have 
the same probability of answering any assessment item successfully independent of his or her linguistic or 
cultural background. Stimulus materials and ideas for tasks are solicited from as wide a range of 
contributors as possible. Standards are established regarding the translation and adaptation of items. 
Translations and adaptations are centrally verified and item performance is rigorously tested. 

Linking results over time 

IALS, ALLS and PIAAC have all been designed to provide links to previous surveys and thus allow 
the measurement of changes in the literacy competencies of certain populations over time. IALS was 
designed to report data on scales that corresponded to the three scales that were used to report the results of 
the National Adult Literacy Survey (NALS), conducted in the United States in 1992 (Yamamoto, 1998, 
p.161). ALLS was linked to IALS through the use of a common prose and document scale. PIAAC is being 
designed to provide links to both ALLS and IALS in the domain of literacy and ALL in the domain of 
numeracy. Of the countries participating in PIAAC, 20 participated in IALS and nine participated in ALL. 

Background information to examine antecedents and outcomes of observed skills 

In order to achieve the analytical and policy objectives of the surveys, comprehensive background 
information is collected regarding respondents. This covers the antecedents of literacy and other skills such 
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as educational attainment, family background, linguistic background and outcome variables such as labour 
market status, income, health status, and civic participation. In addition, information is collected on literacy 
and numeracy practices at home and work as well as participation in ongoing education and training and 
the use of and familiarity with ICTs. 

Ensuring data quality 

Compliance with technical standards 


The conduct of large scale household based assessments represents a significant operational challenge. One 
of the major sources of risk to the quality of the data lies in the quality of national administration and 
variability in implementation between countries. Error may be produced in all phases of survey 
implementation - e.g. sampling, data collection, scoring and data processing etc. To reduce error and 
ensure the quality of data, participating countries are expected to comply with standards covering aspects 
of the survey such as sampling, data collection, scoring, and data preparation. Compliance with standards 
is monitored. 

Reporting 

A focus on demonstrating competencies and skills 

IALS and its successors have been conceived within a criterion referenced testing framework. Their 
objective is not just to compare distributions of skills across populations but, most importantly, to provide 
an insight into what a person who achieves a particular test scores can actually do. Assessment tasks are 
conceived as representing a dimension of skill as described in the relevant assessment framework. Tasks 
are grouped by difficulty in proficiency levels. A person with a proficiency score in the range defined by a 
given level can be regarded as having mastered the skills required to undertake tasks at this level of 
difficulty - i.e. he or she is able to successfully complete such tasks most of the time. 5 While she may be 
capable of successfully complete tasks of a higher level of difficult some of the time, the probability of 
success is not sufficient to indicate mastery of such tasks. 

In IALS, five levels of proficiency were defined for each of the three literacy scales (prose, document 
and quantitative literacy). ALLS maintained the same five proficiency levels for the prose and document 
literacy scales as in IALS but made some slight modifications in the descriptions of the levels. Live levels 
were defined for numeracy and four in the case of problem solving. In practice, levels 4 and 5 in the 
literacy and numeracy scales are combined in reporting given the relatively low numbers of both 
individuals and items at the highest level. 

4. Developments between IALS and PIAAC 

Skills Assessed 

One of the major areas in which there has been change between assessments concerns the skill 
domains assessed. This can be seen from Table 1 below which presents the skills domains assessed in 
IALS, ALL and those planned for PIAAC. Shading indicates that the skill domains concerned are 
measured on comparable scales. 


5 A person with a proficiency score at the mid-point of the level will have, on average an 80 percent chance of 
successfully completing tasks at that level. 


9 



EDU/WKP(2009)5 


Table 1: Skills Assessed in IALS, Al 

LL and PIAAC 

IALS 

ALL 

PIAAC 

Prose literacy 

Prose literacy 

Literacy (combined prose and 

Document literacy 

Document literacy 

document) 



Reading components 

Quantitative literacy 




Numeracy 

Numeracy 


Problem solving 




Problem solving in technology 
rich environments 


IALS assessed three separate domains of literacy, prose literacy, document literacy and quantitative 
literacy. Prose literacy was defined as the knowledge and skills needed to understand and use continuous 
texts - information organised in sentence and paragraph formats. Document literacy represented the 
knowledge and skills needed to process documents, or information organized in matrix structures (i.e. in 
rows and columns). The type of documents covered by this domain included tables, signs, indexes, lists, 
coupons, schedules, charts, graphs, maps, and forms. Quantitative literacy covered the skills needed to 
undertake arithmetic operations such as addition, subtraction, multiplication, or division either singly or in 
combination using numbers or quantities embedded in printed material. 

The major change between IALS and ALL was the replacement of the assessment of quantitative 
literacy with that of numeracy and the introduction of the assessment of problem solving. Numeracy 6 
represented a broader domain than that of quantitative literacy, covering a wider range of skills and 
knowledge (not only computational operations) as well as a broader range of situations in which actors had 
to deal with mathematical information of different types (not only situations involving numbers embedded 
in printed materials) (Gal, et al., 2005, p. 15 1). Problem solving was defined as ‘goal-directed thinking and 
action in situations for which no routine solution procedure is available’ (Statistics Canada & OECD, 2005, 
P-16). 

PIAAC will differ from ALL in that (reading) literacy will be assessed on a single scale rather than on 
two separate (prose and document literacy) scales. The measurement framework for literacy in PIAAC 
draws heavily on those used in IALS and ALL but has expanded the kinds of texts covered to include 
electronic texts in addition to the continuous (prose), non-continuous (document) and combined texts of the 
IALS and ALL frameworks. In addition, the assessment of literacy will be extended to include a measure 
of reading component skills. This is designed for people with low levels of literacy competence and will 
focus on assessment of the foundational skills needed to gain basic meaning from texts. The skills tested 
will include print vocabulary, sentence processing, and passage fluency. 

Problem solving in technology rich environments constitutes a new domain. While it has some 
relationship to problem solving as conceived in ALL, the emphasis is on the skills necessary to solve 
‘information problems’. These cover the specific class of problems people deal with when using ICT 
which share the following characteristic: the existence of the problem is mostly a consequence of the 
availability of new technologies; the problem solution requires the use of computer-based artifacts (tools, 
representational formats, computational procedures); and/or he problems are related to the handling and 
maintenance of technology rich environments themselves. Problem solving in technology rich 
environments is defined as the use of ‘digital technology, communication tools and networks to acquire 
and evaluate information, communicate with others and perform practical tasks’. The first PIAAC 


Defined as “the knowledge and skills required to effectively manage and respond to the mathematical demands of 
diverse situations” (Gal, et al., 2005, p. 1 5 1). 
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assessment will focus on the ability to solve problems using a restricted range of applications - the world- 
wide web, spreadsheets and email. 

The other area in which PIAAC will broaden the amount of information collected regards skills use. 
The so-called Job Requirements Approach (JRA) will be used to gain information on the incidence and 
intensity of the use of a broad range of generic skills in the workplace extending beyond the use of literacy, 
numeracy and problem solving skills. The JRA method consists of asking individuals about the importance 
of different types of tasks that they may be performing in their jobs and subsequently inferring the types of 
skills that are required from their answers. This approach has been used in the UK Skills Survey (Felstead, 
et al„ 2007). 

Assessment technology 

PIAAC also differs from IALS and ALL in that it is being developed as an integrated computer-based 
assessment. The background information, including information on skills use will be collected on a 
specially developed CAPI application. The majority of respondents in the majority of participating 
countries will undertake the direct assessment on a laptop computer. The assessment will maximise use of 
the functionality available with computer based testing, particularly the possibility of adaption of testing to 
individual performance and automatic scoring. A pen and paper version of the literacy and numeracy 
assessment will be available for respondents who have insufficient familiarity with computers to 
successfully undertake the test on a computer. 

5. The Output of IALS and ALL and how the data have been used? 

International project literature 

Results from IALS were released in three reports between 1995 and 2000 corresponding to the three 
waves of data collection (OECD & Statistics Canada, 1995, 2000; OECD & Human Resources Canada, 
1997). In addition, a Technical Report (Murray, et al., 1998), documenting and evaluating survey methods 
and operations was also published. A report on the first wave of ALL was published by the OECD and 
Statistics Canada (2004) in two volumes and a volume describing the background and conceptual basis of 
the assessment was published by Statistics Canada (Murray, et al., 2005). Results from the second wave of 
data collection for ALL are available for Australia (ABS, 2008) and New Zealand (Satherley, et al., 2008). 
The ALL Technical Report remains as yet unreleased. Public use data sets were released for both IALS 
and the first round of ALL (Statistics Canada, 2007, 2006). 

National project literature 

Reports and studies presenting national level results in IALS and ALL have been published in most of 
the countries that participated in these surveys. These range from the descriptive to the more analytical. 
While many of these publications have been either prepared or commissioned by national authorities, some 
have been written by independent researchers. A list of reports which provide national results is provided 
in Annex A. 

Canada has been particularly active in exploiting the data from IALS and ALL. A number of studies 
concentrating either exclusively on Canadian data or comparing Canada with selected other countries have 
been undertaken by Statistics Canada 7 . New Zealand 8 , Switzerland 9 , Norway and Slovenia have also been 
active in analyzing data from one or both of these studies. 


7 Most can be accessed using the Statistics Canada website: http://www.statcan.ca . 
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International Project Reports and Documentation 

OECD and Statistics Canada (1995), Literacy, Economy and Society: Results of the First International 
Adult Literacy Survey, Paris and Ottawa. 

OECD and Human Resources Development Canada (1997), Literacy Skills for the Knowledge Society: 
Further Results from the International Adult Literacy Survey, Paris. 

OECD and Statistics Canada (2000), Literacy in the Information Age: final report of the International 
Adult Literacy Survey, OECD and Statistics Canada, Paris and Ottawa 

Statistics Canada and OECD (2004), Learning a Living: First results of the Adult Literacy and Life 
Skills Survey, 2 Volumes, Statistics Canada and OECD, Ottawa and Paris 

Murray, Kirsch and Jenkins (eds) (1998). Adult Literacy in OECD Countries: Technical Report on the 
First International Adult Literacy Survey, National Centre for Education Statistics, Office of 
Educational Research and Improvement, Washington DC. 

Murray, S., Y. Clermont, and M. Binkley (eds) (2005). Measuring Adult Literacy and Life Skills: New 
frameworks for assessment. Statistics Canada, Ottawa. 


Research Literature 

There is a reasonably large research literature which uses IALS and ALL. This covers studies 
undertaken by public agencies as well academic researchers on a commissioned or independent basis. The 
release of IALS results also stimulated a number of articles reviewing the methodology and 
implementation of the survey. Overall, the data from IALS have been exploited to a greater extent than has 
ALL. This is probably a function of the fact that more countries participated in IALS than ALL as well as 
the lengthier period over which data from IALS has been available to researchers. 

Issues addressed 

The remainder of this section provides an overview of the range of studies which have used data from 
IALS and ALL and the kinds of issues that have been investigated. 

Distribution of Skills 

The distribution of skills both between and within countries is a key focus of the international 
comparative reports and national reports 8 9 10 . Considerable variation is found in the mean scores on the 
literacy, numeracy and problem solving scales as well as in the distribution of the population across 
proficiency levels. In most participating countries, significant proportions of the population are found to 


8 See Johnston (2004), Satherley and Lawes (2007, 2008a and 2008b) and Satherley, Lawes and Sok (2008a and 

2008b). The publications relating to ALL can be accessed at 
http://www.educationcounts.govt.nz/publications/assessment . 

9 http://www.adult-literacy.admin.ch/bfs/all/fr/index/02.html 

10 See, for example, Tuijnman (2001) which benchmarks IALS results for the US and Canada against those of other 

countries participating in IALS. 
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have low levels of skill. For example, in all but three of the countries participating in IALS, over 40 
percent of the population is assessed to possess document literacy skills which place them in the two 
lowest levels of proficiency. In five countries the proportion of the population with skills at this level rises 
to over 60 percent (OECD and Statistics Canada, 2000, p. 17). 

For those countries which participated in both IALS and ALL, it is possible to look at change in both 
the level and distribution of prose and document literacy proficiency of the population. Growth in prose 
and documents proficiency is found for German speaking Switzerland and a decline in the case of the US 
for prose literacy and Italian speaking Switzerland for both prose and document literacy (OECD and 
Statistics Canada, 2000, pp. 40-41). New Zealand (Satherley, et al., 2008a) also improved its scores in both 
prose and document literacy between IALS and ALL. Reductions in the range of scores from the 5th to the 
95th percentiles are substantial in all countries and regions, except in Norway. 

Antecedents of Skills 

The relationship between background and other variables which are considered to influence 
performance in literacy numeracy and problem solving and measured performance is examined in many 
studies either as an issue in its own right or in conjunction with other analyses. 

The final summary report of the IALS study (OECD and Statistics Canada, 2000) devotes a chapter to 
a discussion of how literacy is developed and sustained. The relationships between variables considered to 
be determinants of literacy competency and literacy outcomes are explored firstly in isolation and, 
secondly, as part of a multivariate model. In summary, the main findings are the following. Formal 
educational attainment is the main determinant of literacy proficiency. Age, occupation and using a 
language other than the one used for testing are also major determinants of performance. White-collar high 
skilled occupational categories correspond with high literacy, but the higher the age of the respondent, 
other variables equal, the lower the level of literacy. Labour force participation, formal adult education and 
informal learning at work measured by reading practices are associated with literacy proficiency in most 
countries, but compared with the previously mentioned variables, their impact is relatively modest. 

An analysis of the determinants of literacy using data for 18 countries using IALS data is also 
undertaken by Desjardins (2003b). He uses a structural model to investigate the predictive capacity of 
determinants of literacy proficiency related to home background, education, labour force status and literacy 
behaviours. Other things being equal, education is found to be the strongest predictor but its effect is 
mediated through further learning occurring at work, home and in the community. 

Similar outcomes are found for ALL (Statistics Canada and OECD, 2005). In particular, educational 
attainment has a strong positive relationship with literacy, and age has a negative relationship with literacy 
skills which remains when educational attainment is taken into account. Literacy skills are also found to be 
positively related to parental background, engagement in literacy related activities and undertaking the 
assessment in one’s mother tongue. 

Literacy and economic (and other) outcomes 

The relationship between literacy, education and labour market outcomes particularly income is an 
issue that receives considerable attention in the literature analysing data from IALS and ALLS. In addition 
to the project reports, analyses which have looked at the relationship between education, literacy and 
employment, unemployment and earnings outcomes include studies using Australian data (Chisick, Lee 
and Miller, 2002), Canadian data (Bonikowska et al, 2008, Green and Ridell, 2003, 2007, Osberg, 2000), 
Irish data (Denny, et al., 2000), Swiss data (Falter, Pasch and Hertig, 2007) and UK data (McIntosh and 
Vignoles). Other authors have investigated similar issues using data from several countries (Barone and 
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Van de Werfhorst, 2008, Denny, et al., 2004). Literacy skills are found to be positively related to chances 
of employment, the probability of leaving unemployment and to wages. The size of the wage premium 
accruing to literacy proficiency varies between countries (and between studies). In addition, where the 
issue is investigated, a large component of the financial return to education is found to be related to literacy 
levels in most cases. An exception is Switzerland where literacy and numeracy competencies are found to 
have only a small impact on wages. 

IALS data have also been used to examine the role of basic skills in the explanation of differences in 
the levels of earnings inequality between countries. Freeman and Shettkat (2000) combine information 
from the Comparable German American Sectoral Database (CGAS) with data from IALS to examine the 
relation between wages, skills and employment in the US and Germany. Devroye and Freeman (2002) 
examine the possibility that cross-country differences in the distribution of skills determine cross-country 
differences in earnings inequality using IALS data. They find that skill inequality explains only about 7% 
of the cross-country difference in inequality. Blau and Kahn, (2005) use data from IALS for nine countries 
to examine the role of cognitive skill in explaining higher wage inequality in the United States than other 
advanced economies. They find that while performance on cognitive tests plays a role in explaining greater 
U.S. wage inequality, higher labor market prices (i.e., higher returns to measured human capital and 
cognitive performance) are quantitatively considerably more important than differences in the distribution 
of test scores in the US. A much greater role is attributed to the supply of skills in explaining relative 
wages by Leuven et al. (2005) who find that about one third of the variation in relative wages between skill 
groups across countries is explained by differences in net supply of skill groups. 

Desjardins (2003a) investigates the relationship of formal education and other factors (including 
literacy skills) with wages and civic participation in Canada using a structural equation model. He finds 
that initial education has the strongest effect on outcomes, but that this is mediated by a complex set of 
inter-relationships. Formal learning induces individuals to ‘participate in the labour market, improving 
access to highly skilled occupations, engaging in more informal job-related learning and, in turn, 
developing and maintaining skills’ (Desjardins, 2003a, p.30). 

In an example of how data from these surveys can be used in macro-level analysis, Coulombe, 
Tremblay and Marchand (2004) use data from IALS to investigate the relationship between human capital 
and growth across 14 OECD countries. They use the age distribution of the test results to construct a 
synthetic time series, over the period of 1960-1995, of the literacy level of the youth cohort entering the 
labour market in each period. The relative literacy level of these cohorts is seen as an indicator of a 
country’s investment in human capital. They argue that that direct measures of human capital based on 
literacy scores outperform measures based on years of schooling in growth regressions and that human 
capital indicators based on literacy scores have a positive and significant effect on the transitory growth 
path, and on the long run levels of GDP per capita and labour productivity. 

The relationship of literacy to other outcomes such as health and civic and social participation is 
investigated by several authors. The ALL report undertakes a preliminary discussion of the relationships 
between literacy and health status (Statistics Canada and OECD, 2005, pp 249-261). Denny (2003) 
explores the relationship of participation in civic activities with both educational attainment and literacy 
proficiency using IALS data. He finds that both education and literacy have direct effects on participation 
in these activities and that about half of the effect of education is accounted for by literacy performance. 
Roberts and Fawcett (1998) explore the role of literacy in the relationship between the socio-economic 
environment and health status in Canada using data from IALS. People with lower literacy levels are found 
to be more likely to be at highest health risk, with an increased effect in the case of senior citizens. 
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Murray et al (2007) develop a health literacy scale using items from the ALL study to describe the 
distribution of health literacy across regions and demographic groups in Canada as well as to provide 
comparisons with the US. Health literacy is also found to be positively linked to self reported health status. 

Education and Skills Mismatch 

Data from IALS and ALL on measured literacy skills and the use of these skills at work has been used 
to examine issues of skills match and mismatch - i.e. the extent to which the skills individuals possess are 
utilised or not in their work. In particular, these surveys have allowed researchers to extend the mismatch 
literature beyond education mismatch to skills mismatch more generally and using more objective 
measures of literacy skills possessed by individuals versus the literacy skill requirements of their jobs. The 
ALL report devotes a section to skills match and mismatch (Statistics Canada and OECD, 2005, pp. 142- 
146) and Lalter, Pasch and Hertig (2007, pp. 31-47) use ALL data to examine the issue in Switzerland. 
Green, Macintosh and Vignoles (1999 and 2002), and Boothby (2002) also examine issues of skills match 
and mismatch using IALS data. 

Statistics Canada and OECD (2005) report that, in six of the countries participating in the first round 
of ALL, around 60 percent of workers have skills which match their level of engagement with literacy and 
numeracy in their work. A similar figure is reported for Switzerland (Lalter, Pasch and Hertig, 2007, p. 46) 
with individuals with intermediate qualifications most likely to be in a situation of mismatch. Green, 
Macintosh and Vignoles (1999) find that 20 percent of IALS respondents in Britain had literacy skills 
which appeared to be underused in their jobs. In a later (2002) article, the same authors further explore 
mismatches and returns to over and under-education and basic skills. They find that, in Britain, women are 
more likely to over-skilled than men, that part-time workers are more likely to be over-skilled than full- 
timer workers and there is a significant wage penalty associated with over-skilling. Boothby (2002) also 
explores the relationship between over- and under-education and literacy skills in Canada on the basis of 
IALS data. He finds while schooling has the largest effect on occupational assignment, literacy skills are 
also important determinant (e.g. a person with high literacy skills is more likely to be in a job for which 
he/she is ‘under-qualified’ from the point of view of his/her qualifications). 

Adult learning 

Data from both IALS and ALL has been used to analyse the incidence and distribution of adult 
learning. One of the major strengths of IALS and ALL is that they provide one of the few sources of 
harmonised data available on participation by adults in education and training. O’Connell (1999) uses 
IALS to undertake a comparative analysis of participation in continuing education and training by adults in 
1 1 OECD countries with a focus on the incidence and duration of involvement in education and training 
and its relationship to factors such as country, gender, prior educational attainment, and occupation. 
Tuijnman and Boudard (2001) present 15 indicators of participation in adult education comparing the 
functioning of training markets in Canada and the US with those of other advanced countries. 

The determinants of participation in adult education and training are examined by Boudard and 
Rubenson (2003) and Boudard and Mourlaix (2003). Rubenson, et al. (2007) undertake a similar exercise 
drawing on ALL data to look at participation in adult learning, inequality in adult learning and the factors 
determining participation in adult learning both across countries (Canada, Norway, Switzerland and the 
US) and between provinces within Canada. Rates of participation in adult learning are found to vary 
between countries with educational attainment and literacy scores being positively related to participation. 
Initial and continuing education are found to be complements rather than substitutes. Engagement in 
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literacy related activities is also found to be a strong predictor of participation by Boudard and Mourlaix. 11 
Coulombe and Tremblay (2007) approach the issue from a different angle, using IALS data on training and 
literacy in an analysis of the macroeconomic determinants of job-related training activity in fourteen 
OECD countries. They find, consistent with the microeconomic evidence, that average literacy skills in a 
country have a positive effect on the proportion of workers trained. 

Skills gain and loss over the life cycle 

Age is found to be closely related to literacy performance in both IALS and ALL. Particularly after 
the age of 45, performance in basic skills declines rapidly in all participating countries even with controls 
for educational attainment. Individually these surveys do not allow investigation of the extent to which the 
observed relation is linked to cohort period or aging effects. The fact that several countries administered 
both IALS and ALL and that the literacy scales are linked allows a deeper analysis of the effects of aging. 

Willms and Murray (2007) use data from IALS and ALL to explore how Canada’s stock of literacy 
skills evolved between 1994 and 2003. They employ a synthetic cohort analysis 12 to document net changes 
in literacy skills for various demographic groups for Canada and the provinces and to explore the 
individual characteristics that influence whether a particular group has gained or lost skill on average over 
the nine year period between surveys. They find evidence of significant literacy skill loss in adulthood, in 
particular, concentrated in adults from lower socio-economic backgrounds. 

Green and Ridell (2007) also look at the effects of age on literacy by following cohorts in IALS and 
ALL for Canada. In line with Willms and Murray, Green and Ridell find that between 1993 and 2003, 
successive birth cohorts in Canada have had poorer literacy outcomes at the top of the distribution. They 
suggest that this potentially points to ‘an education system that is doing better for those at the low end but 
doing a poorer job of generating literacy for those at the top (2007, p.41). They also find that literacy tends 
to decline with age after leaving formal schooling and that schooling itself is the prime driver of literacy. 

Different patterns of growth and decline in attainment are evident in Australia New Zealand and 
Switzerland. Australian data (ABS, 2008), suggests a decline in literacy skills in the period between 1996 
and 2006 for cohorts aged 34 and over in 1996. In New Zealand, Satherley and Lawes (2008a) report 
growth in the literacy attainment between 1996 and 2006 for all birth cohorts studied. In the case of 
Switzerland (Notter et al., 2006, pp. 36-37), growth in attainment between 1994 and 2003 is observed for 
cohorts other than the youngest (aged 17-24 in 1994) and the oldest (aged 49-56 in 1994). 

Van der Kamp and Boudard (2003) compare the literacy skills of older adults in the Netherlands with 
other age groups, and present an analysis of the determinants of literacy proficiency of older adults. Results 
from follow-up interviews conducted with 40 older Dutch adults who had low proficiency scores in IALS 
are also presented with a focus on the problems they experience in daily life, the coping strategies they 
develop, and their attitude towards educational interventions. They conclude that the lower literacy 
proficiency of older adults gives reasons for concern but that many older adults develop practical coping 
strategies to compensate. 


They argue that ‘ces resultats remettent en question l’idee re 9 lie selon laquelle le niveau d’instruction constitue a 
lui seul la variable la plus importante expliquant la participation a F education des adultes’ (2003, p. 520). [‘These 
findings put in question the commonly held idea that educational attainment is by itself the most important variable 
explaining educational participation by adults.’] 

12 The evolution of the literacy scores of an age cohort over time is examined by comparing the scores of respondents 
aged x years when surveyed in 1994 in IALS with those of respondents aged x+9 years when surveyed in 
2003 in ALL. 
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Immigration 

The cognitive skills and labour market performance of persons with immigration backgrounds is 
examined in several studies. Statistics Canada and OECD (2005) present results for the countries in the 
first round of ALL. Having a mother tongue other than the official language of the receiving country is 
negatively associated with measured literacy. However, both the skill composition of the migrant 
population and their labour market outcomes vary between countries. Looking at IALS data from Canada, 
New Zealand, Switzerland, and the United States Kahn (2004) finds that immigrants have lower literacy 
test scores than natives with considerable variation in the immigrant/native skill gap between countries. 
The largest gap is found in the US with smaller skill gaps existing in Canada and New Zealand. Analyses 
of Canadian data from IALS (Lerrer, Green and Riddell, 2004) and ALL (Bonikowska, Green and Riddell, 
2008) conclude that immigrants have lower test scores than the native born but find no evidence to support 
the hypothesis that immigrants receive lower returns to literacy skills than non-migrants. Low literacy 
among immigrants is acknowledged as an important factor in understanding immigrant-native born 
earnings differentials in Canada but is not the dominant explanation. 

ICT usage 

The relationships between literacy, ICT use and outcomes in countries participating in ALL are 
explored in Statistics Canada and OECD (2005) and in more depth by Veenhof, Clermont and Sciadas 
(2005). Literacy skills are found to be positively related to computer use and combined literacy and 
computer use profiles were strongly related to the likelihood that respondents had higher incomes. 

Aspects of low skill 

Issues relating to the population with low levels of literacy and numeracy skill have been examined in 
several studies. 

Learning difficulties 

Two special issues of the journal Dyslexia (Vol. 2003/9, No. 2 and 4) are devoted to papers using 
IALS data to examine issues related to learning difficulties (see Chapman, et al., 2003; Magajna, et al., 
2003; Ruijssenaars and Ghesquiere, 2003; Vogel, 2003; and Vogel and Holt, 2003. In particular, the 
characteristics and performance of respondents reporting learning difficulties are compared with those of 
respondents without such difficulties in the a number of countries participating in IALS including the 
Netherlands, New Zealand, Slovenia and the US. 

The characteristics of the population with low skills 

The characteristics of the 22 percent of the Swiss population with skills at levels 1 and 2 in ALL are 
investigated by Zuchuat (2007). This group is found to come from a diversity of socio-economic 
backgrounds and, contrary to expectations, the majority of individuals in this group do not belong to ‘at 
risk’ categories. The low skilled are classified into four sub-groups Tight users’, ‘functional users’, well 
‘educated literate people’ and ‘allophones’ (people whose native language is not the test language). The 
latter group, in particular, is constituted by individuals cumulating a number of disadvantages. 

Literacy component skills 

Studies have been undertaken in Canada and the US to more fully investigate the skills of people with 
low levels of literacy proficiency as measured in IALS and ALL. The Canadian study which administered 
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a number of tests to measure basic reading related skills 13 (reading components) to samples of English and 
French speakers who had participated in the ALL assessment is reported in Grenier et al., (2008). The US 
study, which administered the same set of tests together with items from IALS to a sample of students in 
adult education classes, is described in Strucker, Yamamoto and Kirsch (2007). In both studies, strong 
relationships are found between reading component skills and literacy proficiency. The US study also 
concludes that components proficiencies can be used to create meaningful classes or profiles of low skilled 
readers. 

Methodological issues 

There is a reasonably large body of literature which examines the objectives, methodology and 
implementation of IALS. 14 This was, in part, stimulated by attempts to understand the French results. 
Studies were commissioned by the French government (Dickes & Flieller, 1997; Dickes & Vrignaud, 
1995; Redmond, 1995). The European Commission funded a study examining the methodology and 
implementation of IALS (Carey, 2000) which, in addition to chapters reviewing various aspects of the 
methodology used in IALS, reported the results of an experiment in which a sample of participants in the 
original IALS round in several countries were retested to examine the impact of testing protocols on 
results. In addition, the release of the findings from IALS led to critical assessment of its objectives and 
theoretical framework (Darville, 1999; Hamilton & Barton, 2000; Levine, 1998) as well as to an evaluation 
of various aspects of the methodology and implementation of the survey (Blum, et al., 2001; Sticht, 2001). 

6. Impact on Policy 

Leaving aside the problems of evaluating the impact on public policy of surveys such as IALS and 
ALL under any conditions, 15 the fact that so many countries were involved in one or other or both of these 
surveys means that a global assessment of their impact is well beyond the scope of this paper. It is clear, 
however, that results from IALS influenced policy regarding adult literacy in a number of English speaking 
countries, such as England, Ireland, New Zealand, Northern Ireland and Scotland. 

In England, results from IALS played an important role in raising awareness of the low levels of 
literacy and numeracy among the adult population and contributed to the development of the national 
strategy for improving adult literacy and numeracy skills known as ‘Skills for Life’ (Department for 
Education and Employment, 2001). One commentator (Fawcett, 2003, p. 1 1 9) goes so far as to claim that 
‘the IALS played a seminal role in shaping policy and practice for adults in the UK’. Poor results in IALS 
formed the basis as a call for action by the Working Group headed by Sir Claus Moser 16 which had been 
asked by the government to advise on ways in which the English Government's plans for basic skills 
provision for adults could be supported and developed (Working Group on Post-School Basic Skills, n.d.). 
The Moser report went on to recommend the establishment of a national literacy strategy. In Ireland, IALS 


13 

The tests assessed oral vocabulary, real word reading for speed and accuracy, pseudo-word reading for speed and 
accuracy, spelling, short-term memory. 

14 Some of this literature is surveyed in OECD (2008) which includes an extensive bibliography. 

15 As an example of the range of possible effects, the experience of IALS in France appears to have stimulated the 

development of expertise in large scale assessment methodology which was previously lacking (Bottini & 
Vrignaud, 2005, pp. 39-40). 

16 Moser described the fact that just over 22 percent of English adults were at level 1 on the IALS literacy and 

numeracy scales and that England had one of the highest proportions of the population at this level among 
the countries participating in IALS as ‘a shocking state of affairs in this rich country, and a sad reflection 
on past decades of schooling and policy priorities over the years’ (Moser, 1999). 
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‘elevated concerns about the adult literacy problem to centre stage in educational policy’ (Government of 
Ireland, 2000, p.86) and led to the development of a National Adult Literacy Programme. LALS results 
also played a role in stimulating the development of New Zealand’s adult literacy strategy ‘More than 
Words’ (Office of the Ministry of Education, 2001), the ‘Essential Skills for Living’ strategy in Northern 
Ireland (Department for Employment and Learning, 2002) as well as the Scottish adult literacy and 
numeracy strategy (Scottish Executive, 2001). 

7. Conclusions 

In conclusion, two observations are offered. The first is that the history of international adult s ki lls 
surveys in the OECD area is one of the adaptation of a common measurement approach to changing policy 
needs as well as reflection on the successes and failures of individual assessments within a context of tight 
constraints. In large part, the motivation behind the development of both ALL and PIAAC has been an 
interest in extending the range of skills assessed and/or to improve the policy relevance of the measures 
compared to previous surveys. However, the capacity to innovate in this direction has been limited by both 
costs and considerations of feasibility (particularly respondent burden). The outcome is a compromise 
between what can be imagined and what is possible. ALL represents a good illustration of this point. Of 
the seven assessment domains initially considered for inclusion in ALL (prose literacy, document literacy, 
numeracy, problem solving, practical cognition, teamwork and information and communication 
technology), four were maintained and one was redeveloped as an indirect measure of the use of, attitudes 
to and familiarity with ICTs included as part of the background questionnaire. 

The second is that when PIAAC data becomes available in 2013, researchers and policy makers will 
have access to an unprecedented amount of information regarding basic cognitive skills, skills use and 
individuals’ background characteristics and outcomes extending over the period 1993 to 2011. In 
particular, with 29 countries involved, PIAAC will provide an unparalleled comparative data set, the power 
of which will be increased by the fact that it will have links back to LALS and/or ALL in the case of many 
participating countries. 
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Annex A: Reports of National Results for IALS and ALL 

The following table provides bibliographic references to reports and monographs which report on 
national results for countries participating in IALS and/or ALL. There are a range of other publications 
which look at specific issues in particular countries. Some of these (in English or French) are mentioned in 
the text of this paper. 



Survey 

Country 

IALS 

ALL 

Australia 

ABS, 1997 

ABS, 2008 

Belgium (Flemish Community) 

Van Damme et al., 1997 
Verhasselt, 2002 


Canada 

Jones, 1996 

Statistics Canada, 2005 

Chile 

Bravo and Contreras, 2000 


Denmark 

Jensen et al., 2000 


Finland 

Linnakyla et al., 2000 


Hungary (1) 

Kollo, 2009 


Ireland 

Morgan, et al.,1997 


Italy 

Gallina, 2001 

Gallina (ed), 2006a and 
2006b. 

The Netherlands 

Houtkoop, 2000 
Verhasselt, 2002 


New Zealand 

Culligan, n.d 

Satherley, et al., 2008a 

Norway 

Gabrielsen, 2000 

Gabrielsen, et al. 2005 

Northern Ireland 

Sweeney, et al., 2006 


Portugal(2) 

Funda 5 ao Calouste 
Gulbenkian, 1996 


Scotland 

Scottish Executive, 2001 


Slovenia 

Mozina, et al. 1998 



20 




EDU/WKP(2009)5 


Sweden 

Skolverket, 1996 


Switzerland 

Notter et al., 1997 

Notter et al., 2006 

UK 

Carey, 1997 


USA 

Binkley et al., 1997 
Sum et al., 2002 



(1) No Hungarian national report was produced. However, Kollo (2009) devotes a chapter to the Hungarian 
IALS survey and IALS data are used throughout the book. 

(2) Portugal did not formally participate in IALS. However, it undertook an assessment using IALS 
instruments in 1998 as part of an EU sponsored research project. 
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